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[Document Name] CLAIMS 
[Claim 1] 

A moving picture encoding method for performing a multi-frame motion prediction 
with reference to a plurality of picture frames, characterized in that: 
5 the method includes, in reference frames used for the multi-frame motion prediction, a 

frame that has been encoded in a higher picture quality than the other frames of the same 
picture type. 
[Claim 2] 

The moving picture encoding method according to claim 1, characterized in that the 
10 frame encoded in the higher picture quality is a frame to which more code amount is assigned 
than the other frames of the same picture type. 
[Claim 3] 

The moving picture encoding method according to claim 1 or 2, characterized in that 
the frame encoded in the higher picture quality is a frame having a smaller quantizing 
15 parameter than the other frames of the same picture type. 
[Claim 4] 

The moving picture encoding method according to any one of claims 1 to 3, 
characterized in that the frame encoded in the higher picture quality is a P-picture frame. 
[Claim 5] 

20 The moving picture encoding method according to any one of claims 1 to 4, 

characterized in that the frame encoded in the higher picture quality is a B-picture frame. 
[Claim 6] 

The moving picture encoding method according to claim 5, characterized in that, in 
comparison with a final B-picture frame in the continuous B-picture frames, a B-picture frame 
25 prior to the final B-picture frame is encoded in the higher picture quality. 
[Claim 7] 

The moving picture encoding method according to any one of claims 1 to 6, 
characterized in that the frames encoded in the higher picture quality are arranged at constant 
frame intervals. 
30 [Claim 8] 

The moving picture encoding method according to any one of claims 1 to 7, 
characterized in that a frame interval of the frames encoded in the higher picture quality is 
adaptively changed in accordance with differential information and motion information 
between a reference frame and a subject frame to be encoded. 
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[Claim 9] 

A moving picture encoding apparatus for performing a multi-frame motion prediction 
with reference to a plurality of picture frames, characterized in that the apparatus comprises: 

encoding means for encoding at least one frame in a plurality of reference frames in a 
5 higher picture quality than the other frames of the same picture type. 
[Claim 10] 

The moving picture encoding apparatus according to claim 9, characterized in that the 
encoding means assigns more code amount to a frame than the other reference frames of the 
same picture type to encode the frame in the higher picture quality. 
10 [Claim 11] 

The moving picture encoding apparatus according to claim 9 or 10, characterized in 
that the encoding means sets a smaller quantizing parameter for a frame than the other reference 
frames of the same picture type to encode the frame in the higher picture quality. 
[Claim 12] 

15 The moving picture encoding apparatus according to any one of claims 9 to 11, 

characterized in that the encoding means encodes a P-picture frame in the higher picture quality. 
[Claim 13] 

The moving picture encoding apparatus according to any one of claims 9 to 12, 
characterized in that the encoding means encodes a B-picture frame in the higher picture 
20 quality. 

[Claim 14] 

The moving picture encoding apparatus according to claim 13, characterized in that, in 
comparison with a final B-picture frame in the continuous B-picture frames, the encoding 
means encodes a B-picture frame prior to the final B-picture frame in the higher picture quality. 
25 [Claim 15] 

The moving picture encoding apparatus according to any one of claims 9 to 14, 
characterized in that the encoding means arranges the frames encoded in the higher picture 
quality at constant fame intervals. 
[Claim 16] 

30 The moving picture encoding apparatus according to any one of claims 9 to 15, 

characterized in that the encoding means adaptively changes a frame interval of the frames 
encoded in the higher picture quality in accordance with differential information and motion 
information between a reference frame and a subject frame to be encoded. 
[Claim 17] 



3 



A moving picture encoding program causing an information processing apparatus to 
execute encoding by performing a multi-frame motion prediction with reference to a plurality 
of picture frames, characterized in that: 

the moving picture encoding program causing the information processing apparatus to 
5 execute a process of: 

encoding at least one frame in a plurality of reference frames in a higher picture quality 
than the other frames of the same picture type. 
[Claim 18] 

The moving picture encoding program according to claim 17, characterized in that the 
10 moving picture encoding program causes the information processing apparatus to execute a 
process of assigning more code amount to a frame than the other reference frames of the same 
picture type to encode the frame in the higher picture quality. 
[Claim 19] 

The moving picture encoding program according to claim 17 or 18, characterized in 
15 that the moving picture encoding program causes the information processing apparatus to 
execute a process of setting a smaller quantizing parameter for a frame than the other reference 
frames of the same picture type to encode the frame in the higher picture quality. 
[Claim 20] 

The moving picture encoding program according to any one of claims 17 to 19, 
20 characterized in that the moving picture encoding program causes the information processing 
apparatus to execute a process of encoding a P-picture frame in the higher picture quality. 
[Claim 21] 

The moving picture encoding program according to any one of claims 17 to 20, 
characterized in that the moving picture encoding program causes the information processing 
25 apparatus to execute a process of encoding a B-picture frame in the higher picture quality. 
[Claim 22] 

The moving picture encoding program according to claim 21, characterized in that the 
moving picture encoding program causes the information processing apparatus to execute a 
process of, in comparison with a final B-picture frame in the continuous B-picture frames, 
30 encoding a B-picture frame prior to the final B-picture frame in the higher picture quality. 
[Claim 23] 

The moving picture encoding program according to any one of claims 17 to 22, 
characterized in that the moving picture encoding program causes the information processing 
apparatus to execute a process of arranging the frames encoded in the higher picture quality at 
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constant fame intervals. 
[Claim 24] 

The moving picture encoding program according to any one of claims 17 to 23, 
characterized in that the moving picture encoding program causes the information processing 
5 apparatus to execute a process of adaptively changing a frame interval of the frames encoded in 
the higher picture quality in accordance with differential information and motion information 
between a reference frame and a subject frame to be encoded. 
[Claim 25] 

A moving picture data input/output apparatus to/from which moving picture data 
10 encoded by performing a multi-frame motion prediction with reference to a plurality of picture 
frames is input or output, characterized in that: 

the moving picture data which is input or output includes, in reference frames used for 
the multi-frame motion prediction, a frame encoded in a higher picture quality than the other 
frames of the same picture type. 
15 [Claim 26] 

The moving picture data input/output apparatus according to claim 25, characterized in 
that the frame encoded in the higher picture quality in the motion picture data is a frame to 
which more code amount is assigned than the other frames of the same picture type. 
[Claim 27] 

20 The moving picture data input/output apparatus according to claim 25 or 26, 

characterized in that the frame encoded in the higher picture quality in the motion picture data 
is a frame having a smaller quantizing parameter than the other frames of the same picture type. 
[Claim 28] 

The moving picture data input/output apparatus according to any one of claims 25 to 
25 27, characterized in that the frame encoded in the higher picture quality in the motion picture 
data is a P-picture frame. 
[Claim 29] 

The moving picture data input/output apparatus according to any one of claims 25 to 
28, characterized in that the frame encoded in the higher picture quality in the motion picture 
30 data is a B -picture frame. 
[Claim 30] 

The moving picture data input/output apparatus according to claim 29, characterized in 
that, in the motion picture data, in comparison with a final B-picture frame in the continuous 
B-picture frames, a B-picture frame prior to the final B-picture frame is encoded in the higher 
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picture quality. 
[Claim 31] 

The moving picture data input/output apparatus according to any one of claims 25 to 

30, characterized in that, in the motion picture data, the frames encoded in the higher picture 
5 quality are arranged at constant frame intervals. 

[Claim 32] 

The moving picture data input/output apparatus according to any one of claims 25 to 

31, characterized in that, in the motion picture data, a frame interval of the frames encoded in 
the higher picture quality is adaptively changed in accordance with differential information and 

10 motion information between a reference frame and a subject frame to be encoded. 
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[Document Name] DESCRIPTION 

[Title of Invention] Moving picture encoding method, apparatus and program 

[Technical Field] 

[0001] 

5 The present invention relates to a moving picture encoding technique, and in particular 

relates to a moving picture encoding method, apparatus and program for performing a 
multi-frame motion prediction, and an input/output apparatus to/from which moving picture 
data encoded by the encoding method and apparatus is input or output. 
[Background Art] 

10 [0002] 

Figure 1 is a block diagram showing a configuration of a conventional typical encoder 
for encoding a moving picture signal. The encoder shown in Fig. 1 includes a local decoding 
apparatus, and is made up from frequency converter 101, quantization unit 102, variable length 
encoder 103, dequantizer 104, inverse frequency convertor 105, frame memory 106, intra-frame 
15 predictor 107, motion compensator 108, motion estimator 109, buffer 110, and code amount 
controller 111. 
[0003] 

An input picture frame is fed into the encoder and is divided into a plurality of blocks. 
A predicted value by an intra-frame prediction or an inter-frame prediction is subtracted from 

20 the divided block. Here, the intra-frame prediction is a technique for predicting a current 
picture by using a reconstructed area of the current frame to be encoded, and the inter-frame 
prediction is a technique for predicting a current picture by using a picture frame that is 
previously reconstructed. The picture block in which the predicted value by the intra-frame 
prediction or the inter-frame prediction is subtracted is called a prediction error. 

25 [0004] 

Incidentally, a picture frame encoded by applying the intra-frame prediction to all 
blocks in a current frame to be encoded is called an I-picture. A picture frame coded by using 
both the intra-frame prediction and the inter-frame prediction is called a P-picture. Further, in 
the inter-frame prediction, a picture frame coded by referring to a plurality of picture frames, 
30 which are input before and after the current encoded frame, is called a B-picture. 
[0005] 

Generally, I-pictures are set at constant intervals, and a section that is separated by the 
I-picture and includes a plurality of frames is called a GOP (Group Of Picture). Definitions of 
the I-, P-, and B-pictures and the GOP are stipulated by the MPEG which is the moving picture 
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encoding standard in the international standardization. 
[0006] 

Then, the above prediction error is converted into a frequency domain by frequency 
converter 101. The prediction error converted into the frequency domain is quantized by 

5 quantizer 102. The quantized prediction error is entropy-encoded by variable length encoder 
103 and is stored in buffer 110. Buffer 110 outputs the stored occurring code (a bitstream) 
with a certain timing. The quantized prediction error is returned to the original spatial domain 
again as a local decoding process by dequantizer 104 and inverse frequency converter 105. 
Further, the predicted value is added to the prediction error returned to the spatial domain, and 

10 the prediction error is stored in frame memory 1 06 as a reconstructed picture. 
[0007] 

The reconstructed picture that has been stored in frame memory 106 is referred by 
intra-frame predictor 107, motion compensator 108 and motion estimator 109 to produce the 
predicted value. Therefore, the reconstructed picture stored in frame memory is also called a 
15 reference frame. 
[0008] 

The above is the principle operation in the moving picture compression technique. 

[0009] 

In general, in a digital broadcast system, service and the like, the moving picture signal 
20 is controlled about the occurring code amount, i.e., the bit rate thereof for transmission/storage. 
Code amount control unit 111 performs the two steps, as descried later, in order to control the 
bit rate. 
[0010] 

In the first step, code amount control unit 111 sets a target code amount for each frame 
25 in accordance with each picture type. When R is a code amount assigned to a frame that have 
not been encoded in GOP, Np and Nb are numbers of P-pictures and B-pictures, which have not 
been encoded in GOP, respectively, Xi, Xp, Xb are parameters, each of which represents frame 
complexity of each picture, defined by equations (1) to (3), and Kp and Kb are parameters in 
consideration of subjective picture qualities by picture type, target code amounts Ti, Tp, Tb by 
30 picture types are then given by equations (4) to (6). 
[0011] 

Xi = QixCi (1), 
Xp = Qp*Cp (2), 
Xb = QbxCb (3), 



Ti = R/(l + NpxXp/(KpxXi) + Nb*Xb/(KbxXi)) (4), 
Tp - R/(Np + NbxKpxXb/(KbxXp)) (5), and 

Tb = R/(Nb + NpxKbxXp/(KpxXb)) (6). 
Where Ci, Cp, Cb are occurring code amounts of I-, P-, B-pictures that are finally 
5 coded, and Qi, Qp, Qb are average quantization step sizes of I-, P-, B-pictures that are finally 
coded. 
[0012] 

Whenever each frame is encoded in accordance with the first process and the second 
process, which will be described later, code amount R assigned to a frame that has not been 
10 encoded in GOP is updated by equation (7). 
[0012] 

R = R-Ci,p,b (7). 

Further, when a head picture of GOP is encoded, code amount R is initialized by 
equation (8). 

15 R = bit_ratexN/frame_rate + R (8). 

Where, bit_rate is a target bit rate, frame_rate is a frame rate, and N is the number of 
frames in GOP. 
[0013] 

In the second step, in order to coincide code amounts Ti, Tp, Tb assigned to respective 
20 frames, obtained in the first step, with actual occurring code amounts, quantization steps are 
obtained by feedback control in macroblock units in accordance with the virtual buffer capacity 
that is set by each picture type. 
[0014] 

First, prior to encoding of the j-th macroblock, occupation amounts in the virtual buffer 
25 are obtained by picture types in accordance with equation (9). 

di,p,bG) = di,p,b(0) + BQ-1) - Ti,p,bx(j-l)/MBcount (9). 
di,p,b(0) is an initial occupation amount in the vertical buffer, B(j) is an occurring code 
amount from a head to the j-th macroblock in the frame, and MBcount is the number of 
macroblocks in the frame. 
30 [0015] 

When encoding of each frame is finished, initial occupation amounts di,p,b(MBcount) 
in the vertical buffer by picture types are used as initial occupation amounts di,p,b(0) in the 
vertical buffer relative to the next picture. 
[0.016] 
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Then, quantization step size QG) relative to the j-th macroblock is calculated in 
accordance with equation (10). 

QG) = Qi,p,bxdi,p J bG)x31/(10xr) (10), and 

Qi,p,b = Xi,p,b/Ti,p,b (11). 
5 Where r is a parameter used to control the response speed of the feedback loop, which 

is called a reaction parameter, and is represented by equation (12). 

r = 2xbit_rate/frame_rate (12). 

Incidentally, initial occupation amounts di,p,b(0) in the virtual buffer at the start of 
encoding are represented by equations (13) to (15). 
10 di(0)=10xr/31 (13), 

dp(0) - Kpxdi(O) (14), and 

db(0) = Kbxdi(0) (15). 

On the other hand, a moving picture encoding scheme is proposed in which a 
multi-frame motion prediction that enables the motion prediction is incorporated, that is, the 

15 P-picture is predicted from not only the I-picture or the P-picture that is just previously encoded 
but also the frame that is already encoded and the B-picture is predicted from not only the 
I-picture or the P-picture that is just previously encoded but also the B-picture that is already 
encoded. In this scheme, since a high-quality frame that is previously encoded is selected, and 
then the motion prediction can be performed degree of freedom in the motion prediction 

20 increases. 

[Disclosure of Invention] 

[Problems to be solved by the invention] 

[0017] 

However, the conventional moving picture encoding controls the code amount 
25 assigned to each frame only by using the picture type and the complexity of each picture that is 
finally encoded, without using the degree of freedom in the motion prediction according to the 
multi-frame motion prediction. Therefore, according to the conventional technique, in the 
moving picture compression using the multi-frame motion prediction, moving pictures are not 
encoded in a high picture quality by using the multi-frame motion prediction effectively. Such 
30 a technique is desired. 
[0018] 

The present invention is made by considering the above problem. An object of the 
present invention is to provide a moving picture encoding technique of encoding a moving 
picture in a high picture quality by effectively using the multi-frame prediction. 
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[Means for Solving the Problems] 
[0019] 

The first invention for solving the above problem is a moving picture encoding method 
for performing a multi-frame motion prediction with reference to a plurality of picture frames, 
5 characterized in that: 

the method includes, in reference frames used for the multi-frame motion prediction, a 
frame that has been encoded in a higher picture quality than the other frames of the same 
picture type. 
[0020] 

10 The second invention for solving the above problem is characterized in that, in the 

above first invention, the frame encoded in the higher picture quality is a frame to which more 
code amount is assigned than the other frames of the same picture type. 
[0021] 

The third invention for solving the above problem is characterized in that, in the above 
15 first or second invention, the frame encoded in the higher picture quality is a frame having a 
smaller quantizing parameter than the other frames of the same picture type. 
[0022] 

The fourth invention for solving the above problem is characterized in that, in any one 
of the above first to third inventions, the frame encoded in the higher picture quality is a 
20 P-picture frame. 
[0023] 

The fifth invention for solving the above problem is characterized in that, in any one of 
the above first to fourth inventions, the frame encoded in the higher picture quality is a 
B-picture frame. 
25 [0024] 

The sixth invention for solving the above problem is characterized in that, in the above 
fifth invention, in comparison with a final B-picture frame in the continuous B-picture frames, a 
B-picture frame prior to the final B-picture frame is encoded in the higher picture quality. 
[0025] 

30 The seventh invention for solving the above problem is characterized in that, in any 

one of the above first to sixth inventions, the frames encoded in the higher picture quality are 
arranged at constant frame intervals. 
[0026] 

The eighth invention for solving the above problem is characterized in that, in any one 
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of the above first to seventh inventions, a frame interval of the frames encoded in the higher 
picture quality is adaptively changed in accordance with differential information and motion 
information between a reference frame and a subject frame to be encoded. 
[0027] 

5 The ninth invention for solving the above problem is a moving picture encoding 

apparatus for performing a multi-frame motion prediction with reference to a plurality of 
picture frames, characterized in that the apparatus comprises: 

encoding means for encoding at least one frame in a plurality of reference frames in a 
higher picture quality than the other frames of the same picture type. 
10 [0028] 

The tenth invention for solving the above problem is characterized in that, in the above 
ninth invention, the encoding means assigns more code amount to a frame than the other 
reference frames of the same picture type to encode the frame in the higher picture quality. 
[0029] 

15 The eleventh invention for solving the above problem is characterized in that, in the 

above ninth or tenth invention, the encoding means sets a smaller quantizing parameter for a 
frame than the other reference frames of the same picture type to encode the frame in the higher 
picture quality. 
[0030] 

20 The twelfth invention for solving the above problem is characterized in that, in any one 

of the above ninth to eleventh inventions, the encoding means encodes a P-picture frame in the 
higher picture quality. 
[0031] 

The thirteenth invention for solving the above problem is characterized in that, in any 
25 one of the above ninth to twelfth inventions, the encoding means encodes a B-picture frame in 
the higher picture quality. 
[0032] 

The fourteenth invention for solving the above problem is characterized in that, in the 
above thirteenth invention, in comparison with a final B-picture frame in the continuous 
30 B-picture frames, the encoding means encodes a B-picture frame prior to the final B-picture 
frame in the higher picture quality. 
[0033] 

The fifteenth invention for solving the above problem is characterized in that, in any 
one of the above ninth to fourteenth inventions, the encoding means arranges the frames 
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encoded in the higher picture quality at constant fame intervals. 
[0034] 

The sixteenth invention for solving the above problem is characterized in that, in any 
one of the above ninth to fifteenth inventions, the encoding means adaptively changes a frame 
interval of the frames encoded in the higher picture quality in accordance with differential 
information and motion information between a reference frame and a subject frame to be 
encoded. 
[0035] 

The seventeenth invention for solving the above problem is a moving picture encoding 
program causing an information processing apparatus to execute encoding by performing a 
multi-frame motion prediction with reference to a plurality of picture frames, characterized in 
that: 

the moving picture encoding program causing the information processing apparatus to 
execute a process of: 

encoding at least one frame in a plurality of reference frames in a higher picture quality 
than the other frames of the same picture type. 
[0036] 

The eighteenth invention for solving the above problem is characterized in that, in the 
above seventeenth invention, the moving picture encoding program causes the information 
processing apparatus to execute a process of assigning more code amount to a frame than the 
other reference frames of the same picture type to encode the frame in the higher picture 
quality. 
[0037] 

The nineteenth invention for solving the above problem is characterized in that, in the 
above seventeenth or eighteenth inventions, the moving picture encoding program causes the 
information processing apparatus to execute a process of setting a smaller quantizing parameter 
for a frame than the other reference frames of the same picture type to encode the frame in the 
higher picture quality. 
[0038] 

The twentieth invention for solving the above problem is characterized in that, in any 
one of the above seventeenth to nineteenth inventions, the moving picture encoding program 
causes the information processing apparatus to execute a process of encoding a P-picture frame 
in the higher picture quality. 
[0039] 
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The twenty-first invention for solving the above problem is characterized in that, in 
any one of the above seventeenth to twentieth inventions, the moving picture encoding program 
causes the information processing apparatus to execute a process of encoding a B-picture frame 
in the higher picture quality. 
5 [0040] 

The twenty-second invention for solving the above problem is characterized in that, in 
the above twenty-first invention, the moving picture encoding program causes the information 
processing apparatus to execute a process of, in comparison with a final B-picture frame in the 
continuous B-picture frames, encoding a B-picture frame prior to the final B-picture frame in 
10 the higher picture quality. 
[0041] 

The twenty-third invention for solving the above problem is characterized in that, in 
any one of the above seventeenth to twenty-second inventions, the moving picture encoding 
program causes the information processing apparatus to execute a process of arranging the 
15 frames encoded in the higher picture quality at constant fame intervals. 
[0042] 

The twenty-fourth invention for solving the above problem is characterized in that, in 
any one of the above seventeenth to twenty-third inventions, the moving picture encoding 
program causes the information processing apparatus to execute a process of adaptively 
20 changing a frame interval of the frames encoded in the higher picture quality in accordance 
with differential information and motion information between a reference frame and a subject 
frame to be encoded. 
[0043] 

The twenty-fifth invention for solving the above problem is a moving picture data 
25 input/output apparatus to/from which moving picture data encoded by performing a multi-frame 
motion prediction with reference to a plurality of picture frames is input or output, 
characterized in that: 

the moving picture data which is input or output includes, in reference frames used for 
the multi-frame motion prediction, a frame encoded in a higher picture quality than the other 
30 frames of the same picture type. 
[0044] 

The twenty-sixth invention for solving the above problem is characterized in that, in 
the above twenty-fifth invention, the frame encoded in the higher picture quality in the motion 
picture data is a frame to which more code amount is assigned than the other frames of the same 
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picture type. 
[0045] 

The twenty-seventh invention for solving the above problem is characterized in that, in 
the above twenty-fifth or twenty-sixth invention, the frame encoded in the higher picture quality 
5 in the motion picture data is a frame having a smaller quantizing parameter than the other 
frames of the same picture type. 
[0046] 

The twenty-eighth invention for solving the above problem is characterized in that, in 
any one of the above twenty-fifth to twenty-seventh inventions, the frame encoded in the higher 
10 picture quality in the motion picture data is a P-picture frame. 
[0047] 

The twenty-ninth invention for solving the above problem is characterized in that, in 
any one of the above twenty-fifth to twenty-seventh inventions, the frame encoded in the higher 
picture quality in the motion picture data is a B-picture frame. 
15 [0048] 

The thirtieth invention for solving the above problem is characterized in that, in the 
above twenty-fifth invention, in comparison with a final B-picture frame in the continuous 
B-picture frames, a B-picture frame prior to the final B-picture frame is encoded in the higher 
picture quality. 
20 [0049] 

The thirty-first invention for solving the above problem is characterized in that, in any 
one of the above twenty-fifth to thirtieth inventions, in the motion picture data, the frames 
encoded in the higher picture quality are arranged at constant frame intervals. 
[0050] 

25 The thirty-second invention for solving the above problem is characterized in that, in 

any one of the above twenty-fifth to thirty-first inventions, in the motion picture data, a frame 
interval of the frames encoded in the higher picture quality is adaptively changed in accordance 
with differential information and motion information between a reference frame and a subject 
frame to be encoded. 

30 [Advantageous Effect of the Invention] 
[0051] 

According to the present invention, in the moving picture compression using the 
multi-frame motion prediction, not the picture type and the complexity of each picture that is 
finally encoded are simply used, but code amount control is performed so as to improve the 
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effects of the motion prediction of the whole scene by encoding a frame having high priority as 
a reference frame in a high picture quality in consideration of the relationship between the 
subject frame to be encoded and the reference frame in the multi-frame motion prediction. 
Therefore, in the moving picture compression using the degree of freedom of the motion 
prediction by the multi-frame motion prediction, the present invention can provide the moving 
picture encoding method of the higher picture quality. 
[0052] 

Also, in accordance with the scene of the moving picture, frames to be encoded in a 
higher picture quality are arranged at constant frame intervals or the frame interval of frames to 
be encoded in a higher picture quality are adaptively changed in accordance with differential 
information and motion information between the reference frame and the subject frame to be 
encoded, thereby precisely selecting the reference frame to be encoded in a higher picture 
quality with priority, and thus the code amount control for moving pictures in the higher picture 
quality can be provided. Accordingly, the moving pictures can be encoded in a still higher 
picture quality. 
[0053] 

Also, the more code amount is assigned to the B -picture frame with high priority as a 
reference frame, thereby providing code amount control for moving pictures in a higher picture 
quality. Accordingly, since the effects of the motion prediction of the whole sequence can be 
improved, moving pictures can be encoded in a higher picture quality. 
[Best Mode for Carrying Out the Invention] 
[0054] 

In the present invention, in the moving picture compression using the multi-frame 
motion prediction, not the picture type and the complexity of each picture that is finally 
encoded are simply used, but an encode amount is controlled in a manner that effects of motion 
prediction of whole scene are improved, in consideration of the relationship between a subject 
frame to be encoded and a reference frame in the multi-frame motion prediction, by encoding a 
frame with a high priority as the reference frame in a high picture quality. 
[0055] 

Incidentally, in the following description, "scene" means any number of continuous 
frames. "A frame is encoded in a high picture quality" means that more code amount is 
assigned to the frame, or that the quantization step size of the frame is made small, or that 
coding distortion in the frame is reduced. 
[0056] 
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Hereinafter, the modes for carrying out the present invention are explained as 
compared with the conventional art. 
[0057] 

According to the present invention, in the moving picture encoding method using the 
multi-frame motion prediction, picture qualities of frames of the same picture type are not even 
but variable, thereby enhancing the effects of the motion prediction of the whole scene to 
provide an encoding technique of a moving picture in a high picture quality. 
[0058] 

As described above, in the conventional moving picture encoding method using the 
motion prediction method, because the motion prediction is performed only by the picture that 
has been just encoded, only the encoding can be performed in which the picture qualities of 
frames of the same picture type are kept even. As its reason, when the picture qualities in the 
same scene are varied, a difference between frames increases in a frame that refers to another 
frame in the lowered picture quality and the effects of the motion prediction are lowered. 
Therefore, the encoding efficiency of the whole scene is lowered, and thus the picture quality of 
the moving picture that is encoded is lowered. 
[0059] 

Next, this problem is explained with reference to Fig. 2. 

[0060] 

In the conventional motion prediction shown in Fig. 2, each P-picture refers to only the 
I- or P-picture that has been just encoded. Even if P 2 , P4 frames are tried to be encoded in a 
higher quality than Pi, P 3 frames, Pi frame is encoded in the low picture quality, and therefore 
the effects of the motion prediction in P 2 frame that refers to Pi frame are significantly lowered 
and the picture quality in P 2 frame is also significantly lowered. The picture quality in P 3 
frame that refers to P 2 frame is also significantly lowered, and because the picture quality 
lowering propagates in the following frames, the picture quality of the whole scene is lowered. 
Therefore, in the method using the conventional motion prediction, the picture quality of 
pictures of the same picture type must be kept even. 
[0061] 

However, like the coding method to which the present invention is applied, there is a 
multi-frame motion prediction that can perform a motion prediction, by referring to not only the 
frame that has been just encoded, but also the frames that have been already encoded. In the 
case of Fig. 2, according to the multi-frame motion prediction, predictions of dot lines can be 
performed in addition to the conventional method (i.e., a frame to be referred can be selected by 
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an area or by a block in the frame). 
[0062] 

With this multi-frame prediction arrangement, it is possible to perform the encoding 
that varies the picture quality in spite of the same picture type. The present invention is 
5 explained with reference to Fig. 2. 
[0063] 

In the multi-frame motion prediction of Fig. 2, each P-picture refers to a plurality of I- 
or P-pictures that have been just encoded. In this example, two I- or P-pictures are referred. 
Even if Pi frame is encoded in the low quality when P 2 , P 4 frames are tried to be encoded in a 
10 higher quality than Pi, P 3 frames, in P 2 frame that refers to not only Pi frame but also I 0 frame, 
the effects of the motion prediction are not significantly lowered. Therefore, P 2 frame is 
encoded in a higher quality. 
[0064] 

Also, the effects of the motion prediction in P 3 frame that refers to P 2 frame encoded in 
15 the high picture quality are enhanced, and P 3 frame is encoded in a higher quality than the 
conventional art. Further, since P 4 frame refers to P 2 , P 3 frames that are encoded in the high 
picture quality, P 4 frame is encoded in a higher quality. In the following frames, the effects of 
the motion prediction are enhanced and moving pictures can be encoded in a high picture 
quality, similarly to the above frames. 
20 [0065] 

With this arrangement, the effects of the motion prediction are enhanced rather than the 
conventional art, and moving pictures can be encoded in a high picture quality. Also, the 
moving pictures that are encoded according to the present invention subjectively provide the 
higher picture quality than the conventional art, because the frames that are encoded in the high 
25 picture quality are periodically displayed when being decoded, and by persistence of vision, i.e., 
by the human visual feature. 
[0066] 

In the above description, the example using only I- and P-picture is shown, however, 
the present invention is available to a case of using I-, P-, and B-pictures are used. 
30 [0067] 

With reference to Fig. 3, the problem of the conventional art and the solutions by the 
present invention will be explained. Incidentally, explanations are given below while paying 
attention to only B-picture because the basic concept relative to P-picture is similar to that 
shown in Fig. 2. 
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[0068] 

In the conventional art, the B-picture frame is a frame not to be referred, i.e., a frame 
that is not referred from another frame, and the effects of the motion prediction in respective 
B-pictures, which are continuous, are equal. Therefore, the same code amount is assigned to 
the respective B-frames, which are continuous, and the B-frames are encoded in the same 
picture quality. As the reason in that the effects of the motion prediction of respective 
B-pictures, which are continuous, are made equal, when attention is given to Bi and B 2 frames 
in the example of Fig. 3, distances from I 0 , P3 frames to Bi frame are respectively one and two, 
and distances from I 0 , P3 frames to B 2 frame are respectively two and one, and both totals of 
frame distances for determining the effects of the motion prediction are three. 
[0069] 

However, in the moving picture encoding that uses the multi-frame motion prediction, 
a B-picture can also refer to another B-picture that is previously encoded. Therefore, the 
effects of the motion prediction of continuous B-pictures are variable. When an attention is 
given to Bi and B 2 frames in Fig. 3, B 2 frame can perform the motion prediction that refers to 
not only I 0 frame but also Bi frame having a short inter-frame distance, and B 2 frame can 
provide higher performance of the motion prediction than Bi frame, as is apparent. Therefore, 
even if the same code amount is not assigned to Bi and B 2 frames, Bi and B 2 frames can be 
encoded in the same picture quality. 
[0070] 

With this technique, in the moving picture encoding method using the multi-frame 
motion prediction, the present invention provides a moving picture encoding method in a high 
picture quality by distributing a code amount to a leading reference B-frame by priority in order 
of encoding in continuous B-frames. Thus, regarding Bi and B 2 frames in FIG. 3, as the code 
mount is assigned to Bi frame by priority, Bi frame is encoded in the higher picture quality than 
the conventional art. B 2 frame, which refers to Bi frame encoded in the higher picture quality, 
can be encoded in a higher picture quality. In other words, by varying picture qualities of 
B-picture frames, the effects of the whole motion prediction are enhanced and moving pictures 
can be encoded in a higher picture quality. 
[0071] 

Explanations are given below of concrete Examples of the present invention. 
[Example 1] 
[0072J 

Fig. 4 is a block diagram of the present invention in Example 1. The present 
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invention further includes picture quality control unit 112 in addition to the conventional art. 
The operation of code amount control unit 111 in the present invention is different from the 
conventional art based on picture quality control information supplied from picture quality 
control unit 112. 
5 [0073] 

In Example 1, the operation is explained when I-picture and P-picture are used as 
picture types used to encode frames. 
[0074] 

Incidentally, in the following description, explanations are given of picture quality 
10 control unit 112 and code amount control unit 111, which are difference between Example 1 
and the conventional art shown in Fig. 1. 
[0075] 

Picture quality control unit 112 includes picture quality determination unit 1121 and 
picture quality control counter 1 122. 
15 [0076] 

Fig. 5 is a block diagram of picture quality control unit 112. 

[0077] 

The picture quality control information supplied from picture quality control unit 112 
includes high picture quality flag HQ flg, number of remaining high picture quality frames 
20 R_HQ_num, and remaining high picture quality frame number R_HQ_frame_num. 
[0078] 

Picture quality determination unit 1121 in picture quality control unit 112 calculates 
high picture quality flag HQ_flg, number of high picture quality frames RHQjium, and high 
picture quality frame number R_HQ_frame_num. The high picture quality flag is supplied to 
25 code amount control unit 111. The number of high picture quality frames and the high picture 
quality frame number are supplied to picture quality control counter 1122. 
[0079] 

The operation of picture quality determination unit 1121 according to Example 1 is 
explained with reference to Fig. 6. Fig. 6 is a flowchart illustrating the operation of picture 
30 quality determination unit 1 1 2 1 in the present embodiment. 
[0080] 

In the following explanation, an interval of frames to be encoded in a high picture 
quality is represented by S, a frame number that has been just determined to be encoded in a 
high picture quality is represented by prev_fnum (initial value = 0), and the number of frames 
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that have been determined to be in a high picture quality is represented by i (initial value = 0). 
Additionally, frame interval S is not more than MAXJREF which is an interval between a 
subject frame to be encoded and the previous-most frame that can be referred by the 
multi-frame prediction. The operation in each step will be explained. 
5 [0081] 

Incidentally, frame interval S that is encoded in a high picture quality may be changed 
in accordance with a motion speed or a frame rate after compression by a certain period (by 
GOP or by a scene). By selecting frame interval S by GOP or by scene so as to be suitable 
thereto, the encoding efficacy according to the present invention is further improved. 
10 [0082] 

In step SI 01, picture quality determination unit 1121 calculates number of high picture 
quality frames HQ_num by using equation (16) based on frame interval S and number N of 
frames in GOP. 

HQ_num = (N/S) - 1 (16). 
15 In step SI 02, picture quality determination unit 1121 determines whether or not 

number i of frames that have been determined to be in a high picture quality is smaller than 
number of high picture quality frames HQ_num. When smaller, step SI 03 is executed, and 
when not smaller, the process is finished. 
[0083] 

20 In step S103, picture quality determination unit 1121 calculates high picture quality 

frame number HQ_frame_num[i] by using equation (17). Further, number i of frames that 
have been determined to be in a high picture quality is incremented by 1, frame number 
prev_fhum that has been just determined to be encoded in a high picture quality is updated by 
equation (18), and step SI 02 is executed. 

25 HQ_frame_num[i] = prevjiqjium + S (17), and 

prev_fhum = HQ_frame_num[i] (18). 

After the above processes, picture quality determination unit 1121 turns HQ_flag on 
when HQ_num is not less than 1 and turns HQ_flag off when less than 1. 
[0084] 

30 According to the process shown in Fig. 6, the reference frame to be referred is encoded 

in a high picture quality at interval S. Because S is not more than interval MAX_REF of 
previous-most frame that can be referred in the multi-frame prediction, the motion prediction 
can be necessarily performed in all subject frames to be encoded while referring to frames that 
are encoded in the high picture quality. 
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[0085] 

As an example, Fig. 7 shows frames to be encoded in the high picture quality 
according to the proposed method in the case of N=10, MAX_REF=3, and S=2. 
[0086] 

Picture quality control counter 1122 calculates number of remaining high picture 
quality frames R_HQ_num and remaining high picture quality frame number 
R_HQ_frame_num in accordance with frame number coding_frame_num supplied from code 
amount control unit 111 and number of high picture quality frames HQjium and high quality 
frame number HQ_frame_num[HQ_num] supplied from picture quality determination unit 
1 122, and outputs them to code amount control unit 111. 
[0087] 

Picture quality control counter 1 122 operates only when number of high picture quality 
frames HQ_num is not less than 1 . Here, the frame number of I-picture at the head of GOP is 
represented by frame_num_I, high picture quality frame counter HQ_frame_count is set to 0, 
and the operation of picture quality control counter 1122 in the present embodiment is 
explained with reference to Fig. 8. Fig. 8 is a flowchart corresponding to the operation of 
picture quality control counter 1 122 in Example 1 . 
[0088] 

In step S201, picture quality control counter 1122 adds frame_num_I to high picture 
quality frame number HQ_frame_num[HQ_frame_count] supplied from picture quality 
determination unit 1121. This operation is performed to synchronize frame number 
coding_frame_num in being encoded, supplied from code amount control unit 111, with the 
frame number to be encoded in a high picture quality. 
[0089] 

In step S202, picture quality control counter 1122 takes 
HQ_frame_num[HQ_frame_count] as remaining high picture quality frame number 
RHQframenum and outputs (HQ_num - HQ_frame_count) as number of remaining high 
picture quality frames R_HQ_num. 
[0090] 

In step S203, picture quality control counter 1122 determines whether or not frame 
number coding_frame_num is synchronized with high picture quality frame number 
HQ_frame_num[HQ frame_count]. When they are synchronized, step S204 is executed, and 
when they are not synchronized, step S202 is executed. 
[0091] 
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In step S 204, picture quality control counter 1122 increments HQ_frame_count by 1. 
When HQ_frame_count is smaller than HQ_num, the process is returned to step S201, and 
when not smaller, the process is finished. 
[0092] 

Code amount control unit 111, as shown in Fig. 9, frame code amount assignment unit 
1111 and quantizing parameter update unit 1112. The difference between Example 1 and the 
conventional art in code amount control unit 111 is only in the operation of frame code amount 
assignment unit 1111. 
[0093] 

Frame code amount assignment unit 1111 calculates an assigned code amount for each 
frame by using the picture quality control information supplied from picture quality control unit 
1 12 and supplies the assigned code amounts to quantizing parameter update unit 11 12. 
[0094] 

Quantizing parameter update unit 1112 calculates a quantizing parameter by using the 
frame assignment code amount supplied from frame code amount assignment unit 1111 and the 
occurring code amount supplied from buffer 110. 
[0095] 

Frame code amount assignment unit 1111 performs frame code amount assignment 
using high picture quality flag HQ_flg, number of remaining high picture quality frames 
R_HQ_num, and remaining high picture quality frame number R_HQ_frame_num in the 
picture quality control information supplied from picture quality control unit 112. 
[0096] 

The operation of frame code amount assignment unit 1 1 1 1 is explained below. 

[0097] 

In the following explanations, it is assumed that target code amounts Ti, Tp are target 
code amounts by picture type, R is a code amount assigned to frames that have not been 
encoded in GOP, Np is the number of P-pictures that have not been encoded in GOP, Xi is 
screen complexity of I-picture that is finally encoded, Xp is screen complexity of P-picture that 
is finally encoded, and Kp is a parameter by picture type in consideration of the subjective 
picture quality. 
[0098] 

Fig. 10 is a flowchart showing the operation of frame code assignment unit 1111 in 
Example 1. 
[0099] 



23 



In step S301, frame code amount assignment unit 1111 determines whether the high 
picture quality flag supplied from picture quality control unit 112 is turned on or off. When 
the high picture quality flag is turned on, step S302 is executed, and when off, step S307 is 
executed. 
5 [0100] 

In step S3 02, it is determined whether or not the frame to be being encoded is I-picture. 
When the subject frame to be encoded is I-picture, step S303 is executed, and when not 
I-picture, step S304 is executed. 
[Q101] 

10 In step S303, frame code amount assignment unit 1111 calculates code amount Ti for 

I-picture that is being coded, by equation (19) and finishes the frame code amount assignment. 
Ti = R/(l + NpxXp/Xi) + additional_Ti (19), 
additional_Ti = residul_bitl xXi/Xgop2 (20), 

residu_bitl = (margin_ratioxRx(Np - R_HQ_num)xXp)/(KpxXgopl) (21), 
15 Xgopl = Xi + NpxXp/Kp (22), and 

Xgop2 = Xi + R_HQ„numxXp/Kp (23). 
Where, margin_ratio is a number not more than 1. 

[0102] 

Since the number of bits more then the conventional art by additional_Ti is assigned to 
20 this frame, the picture quality of this frame is improved. Accordingly, the effects of the frame 
motion prediction that refers to this frame are improved. 
[0103] 

In step S304, frame code amount assignment unit 1111 determines whether or not 
frame number coding_frame_num of P-picture that is being encoded is synchronized with 
25 remaining high picture quality frame number R_HQ_frame_num supplied from picture quality 
control unit 112. When they are synchronized, step S305 is executed, and when not 
synchronized, step S306 is executed. 
[0104] 

In step S305, frame code amount assignment unit 1111 calculates the code amount for 
30 P-picture that is being encoded by equation (24) and finishes the frame code amount 
assignment. 

Tp - R/Np + additionaI_Tp (24), and 

additional_Tp = (margin_ratioxRx(Np - R_HQ_rum))/(NpxR_HQ__num) 
(25). 
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Since the number of bits more then the conventional art by additional_Tp is assigned to 
this frame, the picture quality of this frame is improved. Accordingly, the effects of the frame 
motion prediction that refers to this frame are also improved. 
[0105] 

5 In step S306, frame code amount assignment unit 1111 calculates the code amount for 

P-picture that is being encoded by equation (26) and finishes the frame code amount 
assignment. 

Tp = (1 - margin_ratio)xR/Np (26). 

Since the code amount assigned to the frame is decreased by margin_ratio 5 it is 
10 considered that the picture quality of this frame is lowered. However, since the motion 
prediction can be performed by control of picture quality control unit 1 12 while referring to the 
frames encoded in the high picture quality, the picture quality can be prevented from lowering 
by improvement of motion prediction performance, even if the assigned code amount is small. 
[0106] 

15 In step S307, frame code amount assignment unit 1111 determines whether or not the 

frame that is being encoded is I-picture. When the subject frame to be encoded is I-picture, 
step S308 is executed, and when not I-picture, step S309 is executed. 
[0107] 

In step S308, frame code amount assignment unit 1111 calculates the code amount for 
20 I-picture that is being encoded by equation (27) and finishes the frame code amount 
assignment. 

Ti = R/(l+NpxXp/Xi) (27). 

In step S3 09, frame code amount assignment unit 1111 calculates the code amount for 
P-picture that is being encoded by equation (28) and finishes the frame code amount 
25 assignment. 

Tp = R/Np (28). 

According to the code amount assignment, as described above, all frames can refer to 
the frames encoded in the high picture quality by using the degree of freedom of the motion 
prediction in the multi-frame motion prediction. With this arrangement, since the effects of 
30 the motion prediction of the whole scene are improved, moving pictures can be encoded in a 
high picture quality. 
[0108] 

Quantizing parameter update unit 1112 performs feedback control to a quantizing 
parameter by macroblock, based on the virtual buffer capacity that is set for each picture type, 
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in order to match code amounts Ti,p that are assigned to respective frames and are obtained in 
frame code amount assignment unit 1111 with the actual occurring code amounts. Figure 1 1 is 
a flowchart of updating the quantizing parameter. 
[0109] 

5 In step S401, quantizing parameter update unit 1112 calculates an occupation amount 

of the virtual buffer by picture type with equation (29) prior to encoding of the j-th macroblock. 
di,p© = di,p(0) + BQ-1) - Ti,px(j-l)/MBcount (29). 

Where di,p(0) is an initial occupation amount in the virtual buffer, B(j) is an occurring 
code amount from the head of the frame to the j-th macroblock, and MBcount is the number of 
10 macroblocks in the frame. 
[0110] 

At the completion of encoding each frame, initial occupation amounts di,p(MBcount) 
in the virtual buffer by picture type are used as initial occupation amounts di,p(0) in the virtual 
buffer for the next picture. 
15 [0111] 

In step S402, quantizing parameter update unit 1112 calculates the quantization step 
size relative to the j-th macroblock by equation (30). 

Qstep = Qi,pxdi,p(j)*31/(10xr) (30), and 

Qi,p = Xi,p/Ti,p (31). 
20 With control by frame code amount assignment unit 1111, assignment code amounts 

Ti,p of frames to be encoded in the high picture quality become larger than those of the 
conventional art. Therefore, due to the calculation of equation (31), quantization step sizes 
Qi,p of high picture quality frames are small and the frames are encoded in the high picture 
quality. Accordingly, the average quantizing parameter of the frames of the bitstream to be 
25 output provides small values at frame intervals of S. 
[0112] 

Here, r is a parameter, called a reaction parameter, for controlling the response speed of 
the feedback loop, and is given by equation (32). 

r = 2><bit_rate/frame_rate (32). 
30 Incidentally, at start of encoding, initial occupation amounts di,p(0) of the virtual 

buffer are given by equations (33), (34). 

di(0) = 10*r/31 (33), and 

dp(0) = Kpxdi(O) (34). 

In step S 403, quantizing parameter update unit 1112 detects quantizing parameter Q 
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corresponding to quantization step size Qstep from a quantization table. When there is no 
quantization step size Qstep in the quantization table, quantizing parameter Q of the 
quantization step size closest to quantization step size Qstep is output. 
[0113] 

5 The explanation of Example 1 is completed here. 

[Example 2] 
[0114] 

Example 2 of the present invention is explained. 

[0115] 

10 Fig. 1 2 is a block diagram of the present invention in Example 2. 

[0116] 

It is the characteristic matter of Example 2 that it further includes moving picture 
analysis unit 113 in addition to the arrangement of Example 1. The operation of picture 
quality control unit 112 in Example 2 is different from the conventional art due to frame 
15 differential information and frame motion information both supplied from moving picture 
analysis unit 113. 
[0117] 

Incidentally, in Example 2, the operation of the invention is explained for a case where 
I-picture and P-picture are used as picture types to encode frames. 
20 [0118] 

In addition, in the following description, explanations are given of moving picture 
analysis unit 112 and picture quality control unit 112, which are difference between Example 2 
and Example 1 . 
[0119] 

25 Moving picture analysis unit 113 includes input frame buffer 1131 and motion 

detection unit 1 132. Fig. 13 is a block diagram of moving picture analysis unit 113. 
[0120] 

Moving picture analysis unit 113 calculates the frame differential information and the 
frame motion information from the input picture, and supplies the calculated information to 
30 picture quality control unit 112. 
[0121] 

Each input frame buffer 1131 and motion detection unit 1132, which constitute moving 
picture analysis unit 113, will be explained below. 
[0122] 
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Input frame buffer 1131 foresees N-pieces of input moving picture frames, allocates 
numbers to respective frames in ascending order, and stores the frames in a buffer. The 
number of frames to be foreseen is arbitrary, and, for example, it can be considered that the 
GOP interval used in MPEG is used as N. 
5 [Q123] 

Motion detection unit 1132 calculates the frame motion information and frame 
differential information of each frame in accordance with the frames stored in input frame 
buffer 1131. 
[0124] 

10 The operation of motion detection unit 1132 is explained with reference to Fig. 14. 

Fig. 14 is a flowchart showing the operation of motion detection unit 1132 in the present 
embodiment. Motion detection unit 1132 performs steps described below for each frame, 
which is foreseen. 
[0125] 

15 In step S501, motion detection unit 1 132 divides the frame into blocks of wxh in size 

smaller than the frame size of W*H. 
[0126] 

In step S 502, motion detection unit 1132 calculates block motion information MVX, 
MVY and block differential information D of the blocks to form the frame in accordance with 
20 the following process. 
[0127] 

A pixel value at coordinate (x } y) in the cur-th frame is set as F(cur, x, y), the k-th 
divided block in the cur-th frame is set as B(cur, k), and a coordinate at the upper-left corner of 
B(cur, k) in the frame is set as (bx(cur, k), by(cur, k)). 
25 [0128] 

Block motion information MVX(cur, k, ref), MVY(cur, k, ref) and differential 
information D(cur, k, ref) of block (cur, k) in the cur-th frame, relative to a previous frame by 
ref-pieces, i.e., (cur - ref)-th frame, are given by mvx, mvy that minimize diff(ref, mvx, mvy) 
in equation (35) and minimized diff(ref, MVX(cur, k, ref), MVY(cur, k, ref)). 

30 

diff(rBf,mvx,mvy)' * abs (cur _ pixel (i, j)-ref _pixel(ref,i+mvx, j+mvy)) (35), 

i-o, ho 

cur_pixel(i, j) = F(cur, bx(cur, k) + i, by(cur, k) + j) (36), and 

ref_pixel(ref, i j) = F(cur - ref, bx(cur, k) + i, by(cur, k+j)) (37). 
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In step S503, motion detection unit 1132 calculates the frame motion information and 
the frame differential information from block motion information MVX, MVY and block 
differential information D. 
[0129] 

5 When the number of blocks in the frame is set as block_num, frame motion 

information FMVj(i) and frame differential information FDj(i) of the i-th frame relative to the 
(i-j)-th frame are given by equations (38) and (39). 

block num-1 

FMVffl = ^(abs{MVX{i, n, j)+ MVY(i, n, j))) (38) ( 

block _num-\ 

FDj(i)= (39). 

10 Motion detection unit 1132 performs the above-mentioned process for each frame i 

(l<i<N-l) and each reference frame ref ( 1 <ref<MAX_REF) and calculates the frame 
differential information and the frame motion information between all the input frames and the 
reference frames. 
[0130] 

15 Picture quality control unit 112 includes picture quality determination unit 1121 and 

picture quality control counter 1122. Fig. 15 is a block diagram of picture quality control unit 
112. 
[0131] 

Picture quality control information supplied from picture quality control unit 112 
20 includes a high picture quality flag, the number of remaining high picture quality frames, and a 
remaining high picture quality frame number. 
[0132] 

The difference of picture quality control unit 112 of Example 2 from picture quality 
control unit 112 Example 1 is only in the operation of picture quality determination unit 1121 . 
25 [0133] 

Therefore, only the operation of picture quality determination unit 1121 according to 
the present embodiment is explained below. 
[0134] 

Picture quality determination unit 1121 calculates the high picture quality flag, the 
30 number of high picture quality frames, and the high picture quality frame number in accordance 
with the frame differential information and the frame motion information supplied from moving 
picture analysis 112. The high picture quality flag is output to code amount control unit 111, 
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and the number of high picture quality frames and the high picture quality frame number are 

output to picture quality control counter 1 122. 

[0135] 

The operation of picture quality determination unit 1121 according to the present 
5 embodiment is explained below with reference to Fig. 16. Fig. 16 is a flowchart showing the 
operation of picture quality determination unit 1121 in Example 2. 
[0136] 

In step S601, picture quality determination unit 1121 calculates inter-frame cost IFC(i) 
of each frame i and reference frame j by using equation (40). a is a value depending on the 
10 average quantizing parameter of the frame. 

IFCj(i) = FDj(i) + axFMVj(i) (40). 

In step S602, picture quality determination unit 1121 calculates minimum inter-frame 
cost MINJFC(i) (l<i<N-l) by using equation (41) and obtains best reference frame 
BEST_REF(i) corresponding to MINJFC(i). Here, MAX_REF is an interval between the 
15 subject frame to be encoded and the previous-most frame that can be referred in the multi-frame 
prediction. 

MIN_IFC(i) = m i n {iFCjii)} (41). 

a^\<,j<MAX _PEF 

In step S603, picture quality determination unit 1121 determines whether or not 
BEST_REF(i) (l<i<N-l) of all frames are 1. As shown in Fig. 17, when BEST_REF(i) of all 

20 frames are 1, a continuous scene is predicted without momentary picture fluctuations, such as a 
scene change and a flash, in N-pieces of analysis sections. Therefore, when all 
BEST_REF(i) are 1, the process is advanced to step S604, and when all BEST_REF(i) are not 1, 
the high picture quality flag is turned off and the number of high picture quality frames is set to 
0, and then the process is finished. 

25 [0137] 

In step S604, picture quality control unit 112 calculates average inter-frame cost 
AVERAGEJFC, inter-frame cost upper limit IFC_UPPER_LIMIT, and inter-frame cost lower 
limit IFC_LOWER_LIMIT in accordance with MINJFC(i) (-=IFCl(i)) of all frames by 
equations (42) to (44). Here, margin_ratio is a value not more than 1 . 

AVERAGE _IFC= ]TM/A/_/FC(/)/(/V - 1) (42), 

30 w 

IFC_UPPER_LIMIT = (1 + margin_ratio)x AVERAGEJFC (43), and 
IFC_LOWER_LIMIT = (1 - marginj-atio)x AVERAGEJFC (44). 
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In step S605, picture quality control unit 112 analyzes whether or not MIN_IFC(i) 
(l<i<N-l) of all frames are within a range between IFC_LOWER_LIMIT and 
IFCJJPPERLIMIT. 
[0138] 

5 When MINJFC(i) of all N-l pieces of picture frames are within the above-mentioned 

range, as shown in Fig. 18, the difficulty of the inter-frame prediction is stable, and when not 
within the range, the difficulty is unstable. 
[0139] 

Therefore, picture quality control unit 112 turns the high picture quality flag on and 
10 advances the process to step S606 when MIN_IFC(i) of all frames are within the 
above-mentioned range in step S605. Otherwise, picture quality control unit 112 turns the 
high picture quality flag off, sets the number of high picture quality frames to 0, and terminates 
the following process. 
[0.140] 

15 In step S606, picture quality control unit 112 adaptively calculates frame interval S to 

be encoded in a high picture quality within SS in accordance with the process, which will be 
described later, while maximum high picture quality frame interval set by GOP or scene is 
represented by SS and is a value not more than the interval between the subject frame to be 
encoded and the previous-most frame that can be referred in the multi-frame prediction. 

20 [0141] 

In this description, maximum high picture quality frame interval SS is suitably set in 
accordance with the motion speed and the frame rate after moving picture compression. 
[0142] 

The number of continuous frames not over IFC_UPPER_LIMIT is represented by 
25 s_frame (initial value = 1), frame number that has been just determined to be encoded in a high 
picture quality is represented by prev_fnum (initial value = 0), and frame number in picture 
quality determination process is represented by i (initial value = 2), and an example of the 
calculation method for the number of high picture quality frames HQ_num (initial value = 0) 
and high picture quality frame number HQ_frame_num[N-l] is shown with reference to Fig. 
30 19. 
[0143] 

Fig. 19 is a flowchart showing inside of step S606 in the picture quality determination 
according to the present embodiment. 
[0144] 
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In step S701, picture quality control unit 112 determines whether or not 
IFCs_frame+l(i) is larger than IFC_UPPER_LIMIT, executes step S702 when IFCs_frame+l(i) 
is larger than IFC_UPPER_LIMIT, and executes step S707 when not larger. 
[0145] 

5 In step S702, picture quality control unit 112 obtains the high picture quality frame 

number in accordance with equation (45), updates prev_fnum, HQ_num, and s_frame in 
accordance with equations (46) to (48), and executes step S706. 

HQ_frame_num[hq_num] = prev_fnum + s_frame (45), 

prev_fnum = HQ_frame_num[hq_num] (46), 
10 HQ_fnum = HQ_fhum + 1 (47), and 

sjrame - 1 (48). 

In step S703, picture quality control unit 112 determines whether or not s_frame is 
equal to SS-1, executes step S704 when equal, and executes step S705 when not equal. 
[0146] 

15 In step S704, picture quality control unit 112 obtains the high picture quality frame 

number in accordance with equation (45) while setting s_frame=SS, updates prevjhum, 
HQ_num, and s_frame in accordance with equations (46) to (48), and increments frame number 
i by 1 . Then, step S706 is executed. 
[0147] 

20 In step S705, picture quality control unit 112 increments sjrame by 1, and executes 

step S706. 
[0148] 

In step S706, picture quality control unit 112 increments frame number i to be 
processed by 1, determines whether or not i is smaller than number N of foreseeing frames, and 
25 returns to step S701 when small and finishes the process when not small. 
[0149] 

With the above-mentioned process, while all the subject frames to be encoded refer to 
the frames that are coded in the high picture quality without exception, the motion prediction 
becomes possible. 
30 [0150] 

Fig. 20 shows an example in that the above-mentioned high picture quality encoding 
frame determination is executed when N=15, REF_NUM=3, and the leading frame, that is, 0-th 
frame, is I-picture. According to the inventive method, frame interval S to be coded in the 
high picture quality is updated adaptively. 
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[0151] 

Incidentally, when HQ_num obtained by the above-mentioned picture quality 
determination process is equal to N-l, namely, when it is determined that all frames are 
encoded in the high picture quality, HQ_num is reset to 0, the high picture quality flag is turned 
off, and the number of high picture quality frames is set to 0. 
[0152] 

By arranging moving picture analysis unit 113 is arranged, it is possible to select the 
frame to be encoded in the high picture quality with accuracy, in consideration of momentary 
picture fluctuations, such as a scene change and a flash, in GOP, i.e., rapid reduction of pixel 
correlation between continuous frames and the effects of large fluctuations in encoding 
difficulty of the scene. Accordingly, moving pictures can be encoded a still higher picture 
quality. 
[0153] 

The example of Example 2 has been explained above. The present invention is not 
limited to the above examples. The present invention can be achieved by a simple 
embodiment using inter-frame cost IFC(i). For example, when the interval between the 
subject frame to be encoded and the previous-most frame that can be referred in the multi-frame 
prediction is MAXJREF, high picture quality frame number R_HQ_frame_num that has been 
just encoded in the high picture quality is regarded as one of reference frames and, in frame 
number (R_HQ_frame_num + m) where m<MAXREF, a frame of which inter-frame cost 
IFC(i) relative to the high picture quality reference frame is smaller than a predetermined 
threshold and which corresponds to maximum m can be set as the next high picture quality 
frame. 
[0154] 

The explanation of Example 2 is completed here. 
[Example 3] 
[0155] 

Example 3 of the present invention is explained. In Example 3, the operation of the 
invention in case that I-picture, P-picture, and B-picture are used as picture types used to 
encode frames will be explained. 
[0156] 

The arrangement in Example 3 is similar to the arrangement in Example 1 . However, 
Example 3 is different from Example 1 in the operations of picture quality control unit 112 and 
code amount control unit 111, because encoding is performed by using B-picture. 
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[0157] 

Hereinafter, explanations are given of the operations of picture quality control unit 112 
and code amount control unit 111. 
[0158] 

5 Picture quality control unit 112 includes picture quality determination unit 1121 and 

picture quality control counter 1 122, similarly to Example 1 . 
[0159] 

In the high picture quality flag, the number of remaining high picture quality frames, 
and the high picture quality frame number of the picture quality control information which is 
10 supplied to code amount control unit 111 from picture quality control unit 112, the number of 
remaining high picture quality frames and the high picture quality frame number relate to 
P-picture frames in Example 1, whereas they relate to B-picture frames in Example 3. 
[0160] 

Accordingly, Example 3 is different from Example 1 only in the operation of picture 
15 quality determination unit 1121 in picture quality control unit 112. Therefore, only the 
operation of picture quality determination unit 1121 in Example 3 will be explained below. 
[0161] 

Fig. 21 is a flowchart illustrating the operation of picture quality determination unit 
1121 in the Example. The calculation procedure for number of high picture quality frames 
20 HQ_num (initial value = 0) and high picture quality frame number HQ_frame_num[HQ_num] 
is explained with reference to Fig. 21, when the number of frames included in GOP is set to N 
and the frame number in picture quality determination process is set to i (initial value = 1). 
[0162] 

In step S801, picture quality determination unit 1121 determines whether or not the 
25 coding type of the i-th frame that is currently to be analyzed is B-picture. When the coding 
type is B-picture, step S802 is executed, and when not B-picture, step S804 is executed. 
[0163] 

In step S802, picture quality determination unit 1121 determines whether or not the 
coding type of the successive (i+l)-th frame is B-picture. When the coding type is B-picture, 
30 step S803 is executed, and when not B-picture, step S804 is executed. 
[0164] 

In step S803, picture quality determination unit 1121 stores the current i-th frame as a 
frame to be encoded in a high picture quality in accordance with equation (49) and updates the 
number of high picture quality frames in accordance with equation (50). 
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HQJramejmm[HQ_num] = i (49), and 

HQ_num = HQ_num + 1 (50). 

In step S804, picture quality determination unit 1121 increments the frame number i to 
be processed by 1 and determines whether or not frame number i is smaller than N. When 
frame number i is smaller than N, the process is returned to step S801, and when not smaller, 
the process is finished. 
[0165] 

After completion of the above processes, picture quality determination unit 1121 turns 
high picture quality flag HQ_flg on when HQ_num is not less than 1, and turns HQ_flg off 
when HQ_num is less than 1. 
[0166] 

With the above process, the frame that is to be referred and is encoded as B-picture is 
regarded as a high picture quality frame number. In other words, the reference frame can be 
encoded in the high picture quality. 
[0167] 

Fig. 22 shows an example of B-picture encoded in the high picture quality according to 
such a process. 
[0168] 

Code amount control unit 111 includes frame code amount assignment unit 1111 and 
quantizing parameter update unit 1112, similarly to Example 1. Explanations are given below 
of the operations of frame code amount assignment unit 1111 and quantizing parameter update 
unit 1 1 12 in code amount control unit 111. 
[0169] 

Frame code amount assignment unit 1111 performs frame code amount assignment by 
using the picture quality control information supplied from picture quality control unit 112, i.e., 
the high picture quality flag, number of remaining high picture quality frames R_HQ_num, and 
remaining high picture quality frame number R_HQ_frame_num. 
[0170] 

Target code amounts Ti, Tp, Tb are regarded as target code amounts for respective 
picture types, R is regarded as a code amount assigned to frames that have not been encoded in 
GOP, Np and Nb are regarded as numbers of P-pictures and B-pictures that have not been 
encoded in GOP, Xi, Xp, Xp are regarded as screen complexity of each picture that is finally 
encoded, and Kp and Kb are parameters in consideration of subjective picture quality by picture 
type. The operation of frame code amount assignment unit 1111 in the present Example is 
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explained with reference to Fig. 23. 
[0171] 

In step S901, frame code amount assignment unit 1111 determines whether high 
picture quality flag HQ_flg supplied from picture quality control unit 112 is turned on or off. 
5 When high picture quality flag HQJlg is turned on, step S 902 is executed, and when off, step 
S908 is executed. 
[0172] 

In step S902, frame code amount assignment unit 1111 determines the frame type of 
the frame, which is being encoded. In a case of I-picture, step S903 is executed, in a case of 
10 P-picture, step S904 is executed, and in the other case, i.e., in the case of B-picture, step S905 is 
executed. 
[0173] 

In step S903, frame code amount assignment unit 1111 calculates the code amount for 
I-picture, which is being encoded, in accordance with equation (51), and finishes the frame 
15 code amount assignment. 

Ti = R/(l + NpxXp/(KpxXi) + Nb><Xb/(KpxXi)) + additionalJTi (51), 
additional_Ti = residu_bit3 xXi/Xgop4 ( 52 )> 
residu_bit3 = (margin_ratioxRx(Nb - R_HQ_num)xXb)/(KbxXgop3) (53), 
Xgop3 - Xi + NpxXp/Kp + NbxXb/Kb (54), 

20 and 

Xgop4 = Xi + NpxXp/Kp + (Nb - R_HQ_num)xXb/Kb (55). 

Since the more bits are assigned to this frame by additionalJTi, in comparison with the 
conventional art, the picture quality of this frame is improved. Accordingly, the effects of the 
frame motion prediction that refers to this frame are improved. 
25 [0174] 

In step S904, frame code amount assignment unit 1111 calculates a code amount for 
P-picture, which is being encoded, in accordance with equation (56), and finishes the frame 
code amount assignment. 

Tp = R/(Np + NbxKpxXb/(KbxXp)) + additional_Tp (56), 
30 additional_Tp - residuJ>it4xXp/(KpxXgop6) (57), 

residu_bit4 = (margin_ratioxRx(Nb - R_HQ_num)xXb)/(Kb xXgop5) (58), 
Xgop5 = NpxXp/Kp + NbxXb/Kb (59), and 

Xgop6 = NpxXp/Kp + (Nb-R_HQ_num)xXb/Kb (60). 
Since the more bits are assigned to this frame by additional_Tp, in comparison with the 
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conventional art, the picture quality of this frame is improved. Accordingly, the effects of the 

frame motion prediction that refers to this frame are improved. 

[0175] 

In step S905, frame code amount assignment unit 1111 determines whether or not 
5 frame number coding_frame_num of B-picture, which is being encoded, is synchronous with 
remaining high picture quality frame number R_HQ_framejium supplied from picture quality 
control unit 112. When they are synchronous, step S906 is executed, and when not 
synchronous, step S907 is executed. 
[0176] 

10 In step S906, frame code amount assignment unit 1111 calculates a code amount for 

B-picture, which is immediately coded, in accordance with equation (61), and finishes the 
frame code amount assignment. 

Tb = R/(Nb + NpxKbxXp/(KpxXb)) + additional_Tb (61), and 

additional_Tb = residu_bit4xXb/(KbxXgop6) (62). 
15 Since the more bits are assigned to this additional_Tb in comparison with the 

conventional art, the picture quality of this frame is improved. Accordingly, the effects of the 
frame motion prediction that refers to this frame are improved. 
[0177] 

In step S907, frame code amount assignment unit 1111 calculates a code amount for 
20 B-picture, which is immediately coded, in accordance with equation (63), and finishes the 
frame code amount assignment. 

Tb = (1 - margin_ratio)xR/(Nb + NpxKbxXp/(KpxXb)) (63). 

Since the code amount assigned to this is reduced by margin_ratio, lowering in the 
picture quality of this frame is considered. However, with control of picture quality control 
25 unit 112, the motion prediction can be performed while I-, P-, B-frames that are encoded in the 
high picture quality are referred. Accordingly, even if the code amount is small to some 
degree, lowering in the picture quality is prevented by improvement of the motion prediction 
performance. 
[0178] 

30 In step S908, frame code amount assignment unit 1111 determines the picture type of 

the frame, which is being encoded. In a case of I-picture, step S909 is executed, in a case of 
P-picture, step S910 is executed, and in the other case, i.e., in the case of B-picture, step S911 is 
executed. 
[0179] 
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In step S909, frame code amount assignment unit 1111 calculates the code amount for 
I-picture, which is being encoded, in accordance with equation (64), and finishes the frame 
code amount assignment. 

Ti = R/(l + NpxXp/(KpxXi) + Nb*Xb/(KpxXi)) (64). 

In step S910, frame code amount assignment unit 1111 calculates the code amount for 
P-picture, which is being encoded, in accordance with equation (65), and finishes the frame 
code amount assignment. 

Tp = R/(Np + NbxKpxXb/(KbxXp) (65). 

In step S911, frame code amount assignment unit 1111 calculates the code amount for 
B-picture 5 which is being encoded, in accordance with equation (66), and finishes the frame 
code amount assignment. 

Tb = R/(Nb + NpxKbxXp/(KpxXb) (66). 

With the above code amount assignment, without assigning a residual code amount to 
B-picture frames having low priority as reference frames, frames that are referred and are 
encoded as I-, P-, B-pictures can be encoded in the high picture quality. Accordingly, the 
motion prediction can be performed for all frames by the frames that are encoded in the high 
picture quality, and moving pictures can be encoded in the high picture quality. 
[0180] 

Quantizing parameter update unit 1112 obtains a quantizing parameter according to 
feedback control by macroblock, based on the virtual buffer capacity that is set for each picture 
type, in order to match code amounts Ti,p,b that are obtained in frame code amount assignment 
unit 1 1 11 and are assigned to respective frames with the actual occurring code amounts. 
[0181] 

The operation of quantizing parameter update unit 1 112 is similar to that of Example 1, 
the operation of each step is explained with reference to the flowchart in Fig. 11, in order to 
correspond to a variable name for each picture types. 
[0182] 

In step S401, quantizing parameter update unit 1112 calculates an occupation amount 
of the virtual buffer by each picture type with equation (67) prior to encoding of the j-th 
macroblock. 

di,p,b(j) = di,p,b(0) + B(j-1) - Ti,p,bx(j-l)/MBcount (67). 

Where, di,p,b(0) is an initial occupation amount in the virtual buffer, B(j) is an 
occurring code amount from the head of the frame to the j-th macroblock, and MBcount is the 
number of macroblocks in the frame. 
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[0183] 

At completion of encoding of each frame, the initial occupation amount in the virtual 
buffer for each frame di,p,b(MBcount) is used as an initial occupation amount di,p,b(0) in the 
virtual buffer for the next picture. 
[0184] 

In step S402, quantizing parameter update unit 1112 calculates the quantization step 
size relative to the j-th macroblock by equation (68). 

Qstep = Qi 5 p,bxdi s p 5 b(j)x31/(10xr) (68), and 

Qi,p,b = Xi,p,b/Ti,p,b (69). 

With control by frame code amount assignment unit 1111, assignment code amounts 
Ti,p,b of frames to be encoded in a high picture quality become larger than those of the 
conventional art, and code amount Tb of B-picture that is roughly encoded becomes smaller 
than that of the conventional art. 
[0185] 

According to equation (69), quantization step size Qb of B-picture frames that is 
encoded in the high picture quality becomes small, and quantization step size Qb of B-picture 
frames that is roughly encoded becomes large. In other words, the B-picture frame having 
high priority as a reference frame to be referred is encoded in a higher picture quality than the 
B-picture frame having low priority as the reference frame to be referred. 
[0186] 

Here, r is a parameter used to control the response rate of the feedback loop, which is 
called a reaction parameter, and is represented by equation (70). 
r = 2xbit_rate/frame_rate (70). 

Incidentally, initial values di,p(0) in the virtual buffer at start of encoding are 
represented by equations (71) to (73). 

di(0) = 10xr/31 (71), 
dp(0) = Kpxdi(O) (72), and 

db(0) - Kbxdi(O) (73). 

In step S 403, quantizing parameter update unit 1112 detects quantizing parameter Q 
corresponding to quantization step size Qstep from a quantization table. When there is no 
corresponding quantization step size Qstep in the quantization table, quantizing parameter 
update unit 1112 outputs quantization parameter Q of the quantization step size value closest to 
quantization step size Qstep. 
[0187] 
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The explanation of Example 3 is completed here. 
[Example 4] 
[0188] 

Example 4 of the present invention is explained. In Example 4, the operation of the 
5 invention in case that I-picture, P-picture, and B-picture are used as picture types used to 
encode frames will be explained. 
[0189] 

The arrangement in Example 4 is similar to the arrangement in Example 4. However, 
Example 4 is different from Example 3 in the operations of picture quality control unit 112 and 
10 code amount control unit 111, because encoding is performed by using B-picture. 
[0190] 

Code amount control unit 111 in this Example is the same as code amount control unit 
111 in Example 3. Therefore, only the operation of picture quality control unit 112 in this 
Example is explained below. 
15 [0191] 

Picture quality control unit 112 includes picture quality determination unit 1121 and 
picture quality control counter 1 122, similarly to Example 3. 
[0192] 

The difference of Example 4 from Example 3 is that the operation of picture quality 
20 determination circuit 1121 by the frame differential information and the frame motion 
information supplied from moving picture analysis unit 112 is changed. Therefore, only the 
operation of picture quality determination unit 1 121 in Example 4 will be explained below. 
[0193] 

Fig. 24 is a flowchart showing the operation of picture quality determination unit 1121 
25 according to the present embodiment. 
[0194] 

The calculation procedure for number of high picture quality frames HQ_num (initial 
value = 1) and high picture quality frame number HQ_frame_num[HQ„num] is explained with 
reference to Fig. 24, when the number of frames included in GOP is set to N and the frame 
30 number in picture quality determination process is set to i (initial value = 0). 
[0195] 

In step SI 001, picture quality determination unit 1121 firstly calculates inter-frame 
cost IFCl(i) of each frame i and the reference frame thereof by using frame differential 
information FDl(i) and frame motion information FMVl(i) supplied from moving picture 
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analysis unit 113 in accordance with equation (40). 
[0196] 

Secondly, picture quality determination unit 1121 obtains IFC_UPPER_LIMIT and 
IFC_LOWER_LIMIT in accordance with equations (42) to (44) on the assumption of 
5 MIN_IFC(i)=IFC(i) in equation (42). 
[0197] 

Thirdly, picture quality determination unit 1121 determines whether or not all of 
IFCl(i) satisfy IFC_LOWER_LIMIT<IFC(i)<IFC_UPPER_LIMIT. When all of IFCl(i) are 
within the above range, the process is advanced to step SI 002 while setting i=l, and when not, 
10 the process is finished. 
[0198] 

In step SI 002, picture quality determination unit 1121 determines whether or not the 
coding type of the i-th frame that is currently to be analyzed is B-picture. When the coding 
type is B-picture, step SI 003 is executed, and when not B-picture, step SI 005 is executed. 
15 [0199] 

In step SI 003, picture quality determination unit 1121 determines whether or not the 
coding type of the successive (i+l)-th frame is B-picture. When the coding type is B-picture, 
step SI 004 is executed, and when not B-picture, step SI 005 is executed. 
[0200] 

20 In step S1004, picture quality determination unit 1121 stores the current i-th frame as 

high picture quality frame number HQ_frame_num[HQ_num] in accordance with equation (49) 
and updates number of high picture quality frames HQ_num in accordance with equation (50). 
[0201] 

In step SI 005, picture quality determination unit 1121 increments the frame number i 
25 to be processed by 1 and determines whether or not frame number i is smaller than number N of 
frames. When frame number i is smaller than N, the process is returned to step SI 002, and 
when not smaller, the process is finished. 
[0202] 

After completion of the above processes, picture quality determination unit 1 121 turns 
30 high picture quality flag HQ_flg on when HQ_num is not less than 1, and turns HQ Jig off 
when HQ_num is less than 1. 
[0203] 

Since moving picture analysis unit 113 is arranged, B-picture frames to be encoded in 
the high picture quality can be selected with accuracy, in consideration of momentary picture 
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fluctuations, such as a scene change and a flash, in GOP and the effects of large fluctuations in 
encoding difficulty of the scene. Accordingly, moving pictures can be encoded a still higher 
picture quality. 
[Example 5]. 
5 [0204] 

Example 5 of the present invention is explained. Example 5 is an information 
processing apparatus in which the moving picture encoding apparatus of the present invention 
is implemented. Fig. 25 is a block diagram of a typical configuration of the information 
processing system in which the moving picture encoding apparatus according to the present 
10 invention is implemented. 
[0205] 

The moving picture encoding apparatus according to the present invention can be 
configured by hardware, as is clear from the above explanations, and can be also carried out by 
a computer program. 
15 [0206] 

This information processing system shown in FIG. 25 includes processor 210, program 
memory 202, and storage media 203, 204. Storage media 203, 204 are separate' storage media 
and different storage areas in one storage medium. As a storage medium, a magnetic storage 
medium such as a hard disk drive may be used. 
20 [Example 6] 
[0207] 

Example 6 of the present invention is explained. Example 6 is an input/output 
apparatus to/from which the moving picture data (moving picture bitstream), for example, 
encoded by the moving picture encoding apparatus according to Examples 1 to 6 descried 
25 above is input and output. 
[0208] 

In Example 6, as an example of the input/output apparatus to/from which the moving 
picture data (moving picture bitstream) is input and output, explanations are given of a receiver. 
However, there is no limitation on the input/output apparatus, and the input/output apparatus 
30 may be a frame synchronizer to/from which the moving picture data is input and output, a 
record apparatus such as a video apparatus, and the like. 
[0209] 

Incidentally, in Example 6, explanations are given of a receiver characterized by 
receiving, as inputs, moving picture encoded bit streams generated by Examples 1 and 2. 
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[0210] 

As shown in Fig. 26, the arrangement of the receiver includes reference frame picture 
quality monitor 116 in addition to video decoder 115. The processing in video decoder 115 is 
identical to that in the conventional art. 
5 [0211] 

As shown in Fig. 26, the video decoder (decoder) includes dequantizer 104, inverse 
frequency convertor 105, frame memory 106, inter-frame predictor 107, motion compensator 
108, and buffer 110, which constitute a local decoder in the encoder described in the section of 
Background Art. Further, video recorder 1 1 5 includes variable length decoder 114. 
10 [0212] 

Functions of these elements are similar to those of the encoder described above except 
the variable length decoder. 
[0213] 

The video bitstream mainly includes, as constituent elements, a conversion coefficient 
15 obtained by dividing an original input picture into a plurality of blocks with the encoder and by 
frequency converting and quantizing a prediction error signal, which a predicted value is 
subtracted by the inter-frame predictor or the motion compensator, and a code string, which a 
moving vector for generating the predicted value and the reference frame are variable length 
decoded. 
20 [0214] 

Variable length decoder 114 decodes the variable length code supplied from buffer 110, 
which is stored with the received bitstream, into original values. The decoded values include 
the conversion coefficient, the quantization parameter, the picture type, the motion vector, the 
reference frame, and the like. 
25 [0215] 

Dequantizer 104 dequantizes the conversion coefficient by using the quantizing 
parameter and generates a dequantization conversion coefficient. 
[0216] 

Inverse frequency converter 105 applies inverse frequency conversion to the 
30 dequantization conversion coefficient and generates a prediction error signal. 
[0217] 

The predicted value supplied from inter-frame prediction unit 107 or motion 
compensation unit 108 is added to the prediction error signal, and a decoded picture is obtained. 
[0218] 
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The video decoder repeats these processes, thereby obtaining the decoded picture. 

[0219] 

The decoded picture is stored in frame memory 106 as a reconstructed picture and is 
used to generate the predicted value by inter-frame predictor 107, motion compensator 108, and 
5 motion estimator 109. 

Next, the operation of reference frame picture quality monitor 1 14 is explained. 

[0220] 

Reference frame picture quality monitor 114 monitors the picture type, the reference 
frame, the quantizing parameter, the variable length code, and the frame memory supplied from 

10 video decoder 115, and outputs reference frame picture quality fluctuation information. The 
reference frame picture quality fluctuation flag indicates that the more code amount is assigned 
to the reference frame used for the motion compensation (multi-frame motion prediction) for 
decoding of the moving picture bitstream than other frames of the same picture type or whether 
or not a moving picture coding bitstream includes a frame having a small quantizing parameter. 

15 [0221] 

The reference frame picture quality fluctuation flag is specifically obtained by storing 
the picture type, the code amount, and the average quantizing parameter in decoding of the 
frame that is a reference frame to be referred after decoding and by checking that the more code 
amount is assigned to the reference frame stored in the memory frame than other frames of the 
20 same picture type or is obtained by checking whether or not a moving picture coding bitstream 
includes a frame having a small quantizing parameter. 
[0222] 

Figure 27 shows a flowchart for generating the reference frame picture quality 
fluctuation flag, where the picture type corresponding to reference frame number i 
25 ( 1 <i<MAX_REF) stored in the frame memory is picjype(i), the code amount of the whole 
frame is total_bit(i), and the average quantizing parameter of the whole frame is average_q(i). 
The process in each step is explained below. 
[0223] 

First, in step SA01, reference frame picture quality monitor 114 determines whether 
30 the picture type of the frame, which is being decoded, is P-picture or B-picture. When the 
picture type is P-picture or B-picture, the process is advanced to step SA02, and when neither 
P-picture nor B-picture, the reference frame picture quality fluctuation flag is turned off and the 
process is finished. 
[0224] 
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In step SA02, reference frame picture quality monitor 114 determines that the more 
code amount is assigned to the i-th frame than the j-th reference frame (Condition 1: 
total_bit(i)~(l + margin_ratio)xtotal_bitC), picjype(i) = pic_type(j), and or that the 
quantizing parameter is small (Condition 2: average_q(i)~(l/(l + margin_ratio)xaverage_q(j), 

5 pic_type(i) = pic_type(j), and #j), in a plurality of reference frames ref ( 1 <ref<MAX_REF) of 
the same picture type as the frame, which is being decoded. When Condition 1 or Condition 2 
is satisfied, since the more code amount is assigned to the reference frame used by the moving 
picture bitstream for the motion prediction, than the other frames of the same picture type, or 
since such a reference frame is the moving picture encoded bitstream, reference frame picture 

10 quality monitor 114 turns the reference frame picture quality fluctuation flag on. When 
neither Condition 1 nor Condition 2 is satisfied, the reference frame picture quality fluctuation 
flag is turned off. 
[0225] 

Then, in step SA03, reference frame picture quality monitor 114 inspects the code 
15 length of the variable length code supplied from buffer 110 to calculate code amount 
trnp_total_bit of the whole subject frame to be decoded and calculates average quantizing 
parameter tmp_avaerage_Q of the whole frame from the quantizing parameter by MB supplied 
from variable length decoding 1 14. Also, at this time, picture type tmp _j)c_type supplied from 
variable length decoding 114 is stored. With timing in that the current subject frame to be 
20 decoded is stored in frame memory 106 as number k after completion of decoding, reference 
frame picture quality monitor 1 14 stores tmp_total_bit, tmp_avaerage_Q, and tmp_pcjype into 
pic_type(k), total_bit(k), and average_q(k), respectively, and then the process is finished. 
[0226] 

Incidentally, the above calculation of the reference frame picture quality fluctuation 
25 flag, like this, is performed whenever each frame is decoded. 
[0227] 

With the above processes, when the reference frame picture quality fluctuation flag is 
turned on, it can be confirmed that the more code amount is assigned to the moving picture 
bitstream generated by the encoding method of the present invention, which is the reference 
30 frame used for multi-frame motion prediction, than the other frames of the same picture type, or 
that the frame having the small quantizing parameter is included. 
[Brief Description of the Drawings] 
[0228] 

[Fig. 1] Fig. 1 is a view showing a configuration of a conventional art. 
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[0229] 

[Fig. 2] Fig. 2 is a view for explaining the present invention. 

[0230] 

[Fig. 3] Fig. 3 is a view for explaining the present invention. 

5 [0231] 

[Fig. 4] Fig. 4 is a diagram showing a configuration of Example 1 . 

[0232] 

[Fig. 5] Fig. 5 is a block diagram of a picture quality control unit. 

[0233] 

10 [Fig. 6] Fig. 6 is a flowchart showing picture quality determination. 

[0234] 

[Fig. 7] Fig. 7 is a view for explaining a frame to be encoded in a high picture 

quality. 
[0235] 

15 [Fig. 8] Fig. 8 is a flowchart of a picture quality control counter. 

[0236] 

[Fig. 9] Fig. 9 is a block diagram of a code amount control unit. 

[0237] 

[Fig. 10] Fig. 1 0 is a flowchart of assigning a code amount to a frame. 

20 [0238] 

[Fig. 11] Fig. 1 1 is a flowchart of updating a quantizing parameter. 

[0239] 

[Fig. 12] Fig. 12 is a diagram showing a configuration of Example 2. 

[0240] 

25 [Fig. 13] Fig. 1 3 is a block diagram of a moving picture analysis unit. 

[0241] 

[Fig. 14] Fig. 14 is a flowchart of motion detection. 

[0242] 

[Fig. 1 5] Fig. 1 5 is a block diagram of a picture quality control unit. 

30 [0243] 

[Fig. 16] Fig. 1 6 is a flowchart of picture quality determination. 

[0244] 

[Fig. 1 7] Fig. 1 7 is a view for explaining an optimal reference frame. 

[0245] 
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[Fig. 1 8] Fig. 1 8 is a view for explaining the difficulty of the inter-frame prediction. 

[0246] 

[Fig. 1 9] Fig. 19 is a flowchart of determination of a frame encoded in a high picture 

quality. 
5 [0247] 

[Fig. 20] Fig. 20 is a view for explaining a frame to be encoded in a high picture 

quality. 
[0248] 

[Fig. 21] Fig. 21 is a flowchart of picture quality determination. 

10 [0249] 

[Fig. 22] Fig. 22 is a view for explaining a frame to be encoded in a high picture 

quality. 
[0250] 

[Fig. 23] Fig. 23 is a flowchart of assigning a code amount to a frame. 

15 [0251] 

[Fig. 24] Fig. 24 is a flowchart of picture quality determination. 

[0252] 

[Fig. 25] Fig. 25 is a typical block diagram of an information processing system 
installed with the moving picture encoding apparatus according to the present invention. 
20 [0253] 

[Fig. 26] Fig. 26 is a diagram showing an example of a receiver to which a moving 
picture bitstream generated according to the present invention is supplied. 
[0254] 

[Fig. 27] Fig. 27 is a flowchart showing a process for generating a reference frame 
25 picture quality fluctuation flag. 

[Explanation of Reference Numerals] 

[0255] 

1 0 1 Frequency conversion unit; 

102 Quantization unit; 

30 103 Variable length encoding unit; 

104 Dequantization unit; 

1 05 Inverse frequency conversion unit; 

106 Frame memory; 

1 07 Intra-frame prediction unit; 



108 


Motion compensation unit; 


109 


Motion estimation unit; 


110 


Buffer; 


111 


Code amount control unit; 


1111 


Frame code amount assignment unit; 


1112 


Quantizing parameter update unit; 


112 


Picture quality control unit; 


1121 


Picture quality determination unit; 


1122 


Picture quality control counter; 


113 


Moving picture analysis unit: 


1131 


Input frame buffer; 


1132 


Motion detection unit; 


201 


Processor; 


202 


Program memory; 


203 


Storage medium; 


204 


Storage medium. 



[Document Name] DRAWINGS 
[Fig- 1] 




[Fig- 2] 

Conventional Multi-frame motion prediction 

* motion prediction (Two reference frames) 



I 0 Pi P2 P 3 P* 




I 0 B, B 2 P 3 B 4 B 5 P 6 



3/21 



[Fig. 4] 



f 
O 



6u|poou9 i!}6u8i eiqeuBA 



51 

5 o3 



=s o 



-11 
8 a 



t3 ^ 



El. 



cro 
2>.E 



§1 

fi 



C Q. 



"5 §5 ■« 

8 J 



T,T 



[Fig. 5] 

112 Picture quality control unit 



4/21 



Number of high picture 
quality frames, high picture 
quality frame number 



Picture 
quality 
determination 




Picture quality 








control counter 




Frame number 



1122 



High picture quality flag 



Number of remaining high picture 
quality frames, remaining high 
picture quality frame number 



Picture quality control information 
[Fig. 6] 



(Start of picture quality determination \ 
(HQ_num=0, i=0, prev_hq_num=0) J 



S101 



Calculation of high 
picture quality frames 




End of picture quality^ 
s. determination J 



5/21 

[Fig. 7] 




[Fig. 8] 



c 



Start of operation of 
picture quality control counter 

( H Q_frame_count=0) 



S2Q1 



D 



HQ_frame_numCHQ_frame_count]+=frame_nurn_I 



S202 J 

Output of remaining high picture quality 
frame number and number of remaining 

high picture quality frames 




[Fig. 91 



111 Code amount control unit 



Frame 
assignment 
code amount 



Assignment of 
frame code amount 



Update of quantizing 
parameter 



1112 



Occurring 
code amount 



Frame number Quantizing parameter 



7/21 

[Fig. 10] 
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[Fig. 16] 
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[Document Name] ABSTRACT 
[Abstract] 

[Problem] To provide a moving picture encoding technique of encoding a moving picture in a 
high picture quality by effectively using the multi-frame prediction. 

5 [Solving Means] In the moving picture compression using the multi-frame motion prediction, 
not the picture type and the complexity of each picture that is finally encoded are simply used, 
but code amount control is performed so as to improve the effects of the motion prediction of 
the whole scene by encoding a frame having high priority as a reference frame in a high picture 
quality in consideration of the relationship between the subject frame to be encoded and the 

10 reference frame in the multi-frame motion prediction. 
[Representative Drawing] Fig. 7 



